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ABSTRACT 



The purpose of tiis paper is to describe a new technique for 
using videotaped educational materials that could form an important 
part of a satellite-based educational delivery system- The method 
uses videotapes of regular classroom courses for the instruction of 
a small group of remote students (typically 3 to 10) who are assisted 
by a paraprofessional tutor as they watch the tape. For this rea- 
son the method is called the Tutored Videotape Instruction (TVI) 
technique. 

The paper is developed as follows. We first give a brief 
description of the technique, after which we present a set of guide- 
lines that were used for the development of the program. We then 
describe a number of experiments in which the TVI format has been 
compared to both classroom instruction and "real time" television. 
From these experiments we attempt to isolate the factors that ap- 
pear to be most important for the success of the TVI technique. We 
conclude with an estimate of program costs and a brief assessment 
of the value of developing a satellite-based TVI program. 
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I. INTRODUCTION 

Sometime in the early 1920 *s, shortly after radio broad- 
casting was first proven to be economically feasible, Robert Hutchins 
is said to have predicted that this new technology would undoubtedly 
have a dramatic impact on education. Subsequent events have shown 
that his assessment of the educational potential of radio was proba- 
bly correct, but for a variety of reasons, this potential failed to 
materialize. Later, in the early 1950*s, instructional television 
was introduced with a similar fanfare. However, with a few notable 
exceptions, its potential has aisr failed to materialize; and since 
many of the problems inherent in conventional educational radio and 
television will not be solved by the use of a satellite, it seems 
entirely possible that satellite-based educational delivery could 
also become an expensive failure. It is therefore importar: to be- 
gin with a brief analysis of why it is that technological aids to ed- 
^jcation teldom seem to live up to their potential. 

There is of course a different set of reasons in each case, 
though inconstant financing and the competition with conmiercial in- 
terest are surely among the most pervasive. However, important as 
these factors are, there seems to be a still more basic problem. The 
proponents of media-baspd education describe this problem as a failure 
of the educational establishment to involve itself seriously with 
instructional technology. As a result, they say, the changes in the 
design of the educational system that must be made before instruction- 
al technologies can be used effectively in the educational process 
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have not been forthcoming. This is a valid criticism. However the 
educational establishment makes a counter-argument that is also true: 
effective classroom teachers regularly capitalize on miexpec ted , im- 
planned opportunities for the achievement of specific goals. As Jack- 
son []2] says, "stray thoughts, sudden insights, meandering digres- 
sions and other unpredicted events constantly ruffle the smoothness 
of the instructional dial »gue. In most classrooms, as every teacher 
knows, the path of educational progress could be more easily traced 
by 3 butterfly than by a bullet". Jackson concludes from this that 
e^^uration is best served by tools that can be readily adapted to a 
wido variety of educational tasks with a minimum of advanced plan- 
niru^. Blackboards and books provide, at low cost, an impressive 
standard of flexibility, especially when they are compared to most 
electronics media. Furthermore, despite honest efforts to use elec- 
tronics media over an extended period of time> many teachers have 
still been un a b 1 e to see^ a cl ear improvement in^ learning as a result 
of their rather considerable effort. Hence electronics media are 
generally judged by teachers to be inappropriate educational tools for 
the majority of circumstances. 

If we accept both of these criticisms as valid, we are led 
naturally to seek out those media that can provide the intellectual 
flexibility that is essential for effective education without re- 
quiring a major change in teaching styles; and to apply these media 
to situations where the changes in the educational system that are 
necessary to accomodate the media can be easily made. In this pap- 
er we describe a new technique for using videotaped educational ma- 
terials that seenis to meet all of these criteria. The method makes 
use of unrehearsed, unedited videotapes of regular classroom courses, 
which can be ,>roducad at very low cost. These videotapes are then 
used for the instruction of a small group of students (typically 
3 to 10) who are assisted by a paraprofessi onal tutor as they watch 
the tape. For this reason the method is called the Tutored Videotape 
Instruction technique, and will be referred to as the TVI technique 
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in what follows. It will be clear as the paper enfolds that satel- 
lites could be used to deliver the TVI materials to remote sites 
and to provide the comnunication link between students, tutors and 
teachers that is occasionally required. However, radio and slow- 
scan television, or telephones and the U.S. mail can also pro- 
vide the necessary service, and these alternatives to satellite 
delivery should be carefully considered since the objective is to 
provide the best education at the lowest cost. 

The TVI method was originally developed to provide course work 
in science and engineering to off-campus graduate students at Stan- 
ford University. The technique is based on the common-sense notion 
that students can .earn more effectively if they are given the op- 
portunity to freely stop the lectures to discuss important concepts 
before continuing with the material which the lecturer wishes to pre- 
sent. Data taken over the past three years show that learning is 
most effective when the videotaped lectures are stopped frequently 
(e.g., every 5 to 10 minutes for a period of approximately 3-5 min- 
utes). Interactions of this frequency and duration are of course im- 
possible in a conventional classroom situation. The data also show 
that learning ^'i most effective when a small group of students and 
the tutor water the tape together. A single student, with or with- 
out a tutor, does rot learn as effectively as he/she would in a small 
group. 

The TVI technique responds to the education needs of the stu- 
dents by combining the positi>e features of lectures * .th those of 
small group discussions in which the lectures are used directly to 
provide the basis for discussion. The lectures provide for depth 
and continuity in the subject matter, while the tutorial discussions 
provide a means for making the lectures respond to individual needs 
and differences. Students watching the videotaped lectures feel free 
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to ask quostionr, to botti tno tutor and other students^ and to make spontan- 
eous comments about points or intorc:t in the lecture. In addition, since 
the videotapes jre of an actual classroom, the TVI students hear all of the 
cofTjnents and questions askeJ in class and profit from these exchanges as 
well. In effect the TVI format permits the students and the tutor to manage 
the lecture themselves, and thereby create an intellectually stimulating 
environment v/hich enhances learnincj and creates a positive attitude toward 
both the subject and the group. 

It will perhaps be clear from this brief description that, v^/hile the 
program to be des^^ribed relies heavily on the use of video technology, that 
is probably not its most distinctive feaiijre. Rather, the technology pennits 
us to concentrate attention on some of the most important parameters in the 
learning process and to begin to understand their effects more precisely. 
In fact what has been found would seem to be applicable in a wide variety of 
circumstances. It certainly extends well beyond university lecture courses 
and is in some ways independent of whether technological aids are used in 
the instructional process. 

The program was designed to utilize insofar as possible the experience 
which has been gained over the past two decades concerning the question of 
how technology can be used most successfully in education. An attempt to 
sunmarize this experience led the author to develop a set of guidelines that 
could assist educators in planning for the successful use of technology in 
education [1], Because these guidelines form the basis of the program to 
be discussed, it is appropriate to begin with a brief discussion of them. 

II. GUIDEL£NE$ FOR THE SUCCESSFUL APPLICATION OF TECHNOLOGY TO EDUCATION 

Interest in the use of technological aids to education (particularly 
television) is largely an outgrowth of efforts whose main roots can be traced 
to the late 1940*s and early 1950's. Following the widespread penetration of 
cofifnercial television, the Ford Foundation and later the Federal government 
funded a variety of experiments whose overall purpose was to study the possible 
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impact of television on the educotionnl process. These experinent: were 
organized to Include a complete range of educational institutions from ele- 
mentary school to university and took as their najor emphasis the question 
of whether students could ledrn as well from television as from conventional 
classroom teaching. Surveying the results of 421 experiments carried out 
during the period from 1950 to 19bC, Chu and Schramm [2] concluded that, 
across all grade levels from kindergarten through a Baccalaureate degree, 

and for essentially all types of subject material, students could learn as 

★ 

well from television as from conventional instruction. 

Viewed in its most positive light, this result suggests that access to 
education can be expanded substantially by communications technology, and 
indeed this policy has been urged by educational study groups and implemented 
by both private and public educational institutions for some time, and at 
substantial expense. However, the educational effectiveness of classroom 
Instruction is itself a matter of increasing concern, and in this light the 
"no significant difference" finding does not provide a forceful stimulus to 
promote the application of communications technology to educational problems. 

This latter perspective leads naturally to the question of whether 
educational technology has ever established a clear-cut superiority over in- 
class instruction. Consideration of this question led us to compare some of 
the most successful ventures in educational technology (such as Sesame Street, 
Chicago City TV College, The Open University in Britain, the Australian School 
of the Air, and the Bavarian Telekellog) with some of the least successful 
ventures. From this comparison we attempted to abstract a set of guidelines 
that describe the conditions that seem most likely to insure a successful 
application of technology to education. These guidelines are surely in need 
of further refinement and modification; nevertheless, they do provide a 



Specifically, they found that television was favored in 63 of the ex- 
perimental comparisons, conventional teaching in 50, and 308 showed no sig- 
nificant difference. ^ 

7 



4 



useful starting point for designing new programs. 

The guidelines are as follows: 

1. The educational program should be planned for a specific 
target audience . 

2. Specific educational objectives that are relevant to the 
needs and interests of the target audience must be clearly 
defined. 

3. Technologies should be chosen in terms of the topic to be 
presented. Frequently different technologies are used to 
present different parts of a course, the choic^ being de- 
termined from a consideration of which technology is most 
effective for the material being presented. It is desirable 
(though also expensive) to use both knowledge and media spe- 
cialists to prepare and produce the progra.T:, . 

4. Educators who have a clear interest in learning and using 
the instructional characteristics of various media must be 
selected and trained. 

5. Clear and careful provision for personal interaction is 
important for retention of student interest. 

6. Evaluation and feedback over a period of months or years 
must be used to monitor the educational effectiveness of the 
program and to change instructional materials and methods to 
suit learner needs. 

A brief elaboration of these guidelines is given in the Appendix to this 
paper, where they are used to describe two ventures near the ends of the edu- 
cational spectrum (Sesame Street and the Chicago TV College). In particular 
it is shown there that the guidelines provide some means for estimating the 
time scale and financial resources required to start a new venture. However, 
since our present interest is in describing a new technique for educational 
delivery, we will limit our attention for the time being to the question of 
educational effectiveness. 

From the standpoint of educational effectiveness, the guideline that 
is perhaps most frequently overlooked Is the one relating to personal inter- 
dctioni especially where the use of television Is concerned. Television 
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(including lectures that are videotaped for subseqjont p'lyback) is most 
frequently used as a direct substitute for live lectures in large classes or 
as a means for individuals to view lectures or programs at sites that are 
remote from the point of origination. These applications are developed 
primarily to solve the important problems of cost and access, respectively, 
the assumption being made that courses delivered by television will be as 
effective as conventional education. However, the experiments that led to 
this conclusion tested only factual information gained over a relatively 
brief period of time. Furthermore these experiments identified some charac- 
teristic weaknesses of television as an educational medium. In particular, 
as noted by Schranm [3], it doesn't stop to answer questions; it doesn'c 
readily permit classroom discussion; it is an inefficient medium for conduct 
ing drill; it doesn't adjust very well to individual differences; and it 
tends to encourage a passive form of learning. 

These are very serious weaknesses. They can be removed to some degree 
by using radio talkback to connect remote sites to the television classroom 
[4] or by establishing a regular telephone contact betwee,: students and 
faculty, as the Chicago TV College and others after them have done. However 
these techniques for promoting student-faculty interaction do not provide 
a really effective remedy for the major weaknesses described above. It is 
critically important for students to be able to stop a lecture or a program 
when they have questions. When a lecture is stopped, it is highly desirable 
for it to reniain stopped long enough for the question to be clearly answered 
It Is also highly desirable if the answers to the most important questions 
can be developed as a result of active discussion amongst a small group of 
students. Finally it is important that questions and discussions be used 
to determine background deficiencies of individual students, so that effec- 
tive remedial action can be taken. 

Basically the major weaknesses of 'Mive television" (including video- 
tapes that are simply replayed from start to finish) are all related to 
the fact that it cannot provide the quality of personal interaction among 
both students and faculty that is available in an effective classroom. The 
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concept of the TVI intellectual community, consisting of a small group of 

students, au on-site paraprofessional tutor and a course faculty member, 

was invented in an attempt to circumvent the major deficiencies of televised 

instruction. What we sock to do is to provide small groups of students 

with the high-quality personal interaction that they need to learn effectively. 

What we have found is that the TVI intellectual community can provide a means of 

achieving this goal . 

III. EXPERIMENTS WITH TUTORED VIDEOTAPE INSTRUCTION 

In this section we will describe a number of experiments that we have 
carried out to evaluate the educational effectiveness of the tutored video- 
tape instruction format. All of the experiments to be discussed have been 
concerned with the delivery of engineering and science courses where the 
objectivity of the material makes it relatively easy to measure the effective- 
ness of the method. We believp •:he method can be extended to a much larger 
range of subjects and audiences, though there is insufficient data available 
at present to permit us to define this range of applications with any accuracy. 
Our description of experiments is therefore intended primarily to provide the 
reader with some general ideas about how the TVI format can be set up and 
evaluated. 

The TVI technique was originally conceived as a means of providing 
courses to off-campus engineering students employed at a Hewlett Packard 
plant in Santa Rosa, California, approximately 100 miles north of Stanford. 
The Hewlett Packard management wished to provide the same continuing Honors 
cooperative graduate engineering opportunity that is available to HP employees 
(and those of other companies) at plants near Stanford, where engineering 
courses given on trie Stanford campus are televised to plants within a 50*mile 
radius of the University by the Stanford Instructional Television Fixed Ser- 
vice network. 

The TVI technique was developed to fulfill this need. Since the courses 
of interest were alrr.^Jy being televised, it was a simple matter to video- 
tape the classes live with no further production requirements. It should 
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be noted that an operating TV system is not a prerequisite for the technique . 
to be described, though'it was very helpful in this case because it both 
facilitated the initial experiments und made a thorough comparison between 
the TV students and the videotape students possible. 

It was intended from the outset that satisfactory performance in the 
courses would result in credit toward a Master's degree in Electrical Engi- 
neering from Stanford. Toward this end the Santa Rosa students were required 
to complete the same homework as students on campus, with all papers being 
graded by the same teaching assistant who y ode^ the papers of the on-campus 
and regular TV students. To further facilitate the comparison of student 
performance, the off-campus students were required to come to the campus to 
take the san^r exams as the on-campus students, in the on-campus environment. 

A. Program Design for the Santa Rosa Experiment . The r^ain features of 
the program developed for the Santa Rosa experiment can be most simply de- 
scribed in tenns of the guidelines listed above. 

1. Target Audience - Industrially-based students carrying full 
time job responsibilities. In some cases the students had been studying for 
a Master's degree before they were moved to Santa Rosa, in which case their 
academic qualifications were essentially identical to those of on-campus 
students. However, students at Santa Rosa whose academic credentials were 
known to be inadequate for admission to the Stanford gradurte school were 
also included. 

2. Educational Objectives - The students were known to be inter- 
ested in graduate training leading to a Master's degree. A clearly important 
factor in their success is that their employer shared ^h' . j^, i'^tive, in 
both its financial and academic aspects. 

3. Range of Media Employed . Half-inch reel-to-reel videotapes 
of live classes made by a trained student production staff as the class is 
being conducted. The classroom modifications that are necessary for this 
purpose are described by Pettit and Grace [4]. Basically the studio classrooms 
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are organized to produce a min^'mum impact on the teaching styles and pre- 
ferences of the instructors. Half -inch reel-to-reel equipment is used to 
permit the clear reconstruction of a given TV 'frame at the remote sites. 

Videotapes. were mailed to Santa Rosa along w^th class note?;, homework 
assignments and other materials that were handed out to the on-campus students. 
Videotape: and homework were returned to Stanford in approximately one week. 
On-site tutors contacted the on-campus faculty by telephone aftt.»r the video- 
tapes were watched to discuss problems and obtain supplefnen^|^y material if 
necessary. 

4- Media-sensitive Educational Staf^ . Faculty whose courses are 
to be televised are given a brief training session to familiarize them with 
the capabilities of the television network end suggestions concerning how 
these capabilities can oe used effectively (e.g., effective organization of 
blackboard s^ace, use of a desk pad as an alternative to the blackboard, 
effective preparation of de^nonstrations , and so on). 

In addition, Stanford stef^ members who are responsible for the TV! 
program visit each site to choose the tutors (from among the company*s staff) 
and to instruct them in the use of the videotape as an educational medium. 
An alternative that is presently being explored is to use staff members from 
local educational institutions who have indicated an interest in the program. 

In nearly all cases so far, the tutors have been practising engineers 
without prior experience in teaching. They are chosen primarily on the basis 
of two criteria; viz., (a) a sensitivity to students and an ability to draw 
them Into a fruitful discussion of issues, and (b) a personal interest in 
reviewing the subject to be presented. Other criteria, such as recent ex- 
posure to the course with evidence of high quality performance, were found to 
be significantly less important than the criteria just mentioned. 

The tutor's main functions ?ire (a) to initiate and encourage stopping 
the videotape playback for the immediate resolution of problems; (b) to 
answe** questions that cannot be resolved by the class (if possible); and 
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(c) to contdct the on-campus instructor to obtain answers and supplenicntary 
material If necessary. 

Tutors are also encouraged to visit the on-campus faculs ^cp or twice 
per quarter to become familiar with the course syllabus and jss any 

recurring problems that their students have. It Is also usefu' • point out 
that the tutor Is not responsible for grading homework or assigning course 
grades. His function is simply to help his students learn as effectively 
as he can. 

m 

5. Personal Interaction . Students and tutors aiu* are urged 
to stop the videotape whenever they have problems or questions or whenever 
some particularly Important concept has been presented. Certain obvious 
cues are frequently used by the tutor to Initiate these discussions. For 
example, each time a substantive qu non arises in the on-campus class* the 
tape is stopped and the TVI class attempts to generate the answer before the 
taped lecture proceeds. 

In addition to extensive group interaction during the lectures* students 
are encouraged to discuss problems with each other and with the tutor outside 
the viewing period. Sine* the tutors and students are known to each other 
through their common employment* there is ample opportunity for and little 
Inhibition to such active discussion and mutual assistance. 

6. Evaluation and Feedback . Careful records are maintained of 
TVI student performance on both homework and examinnlions. The performance 
of TVI students is regularly ,ompared to thnt of (i) regular on-campus stu- 
dents in the classroom and (H) 1n'^l off*campus students taking the same 
courses by live television. Thcoe latter students also have an audio talk- 
back connection that permits them to ask questions of the instructor whilo 
the class is actually in progress. 

The homework and test performance of the TVI students 1i analyzed by 
the Stanford TVI staff and the results arc discussed with the tutors by 
telephone. Corrective measures are suggested when necessary. In addition 
course evaluation questionnaires are obtained from the TVI students to 
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assess their attitudes and reactions to the program. 

B. Initial Experimental Results . After the first two quarters of 
operation, the TVI experimental program was evaluated by comparing the course 
performance of the remote TVI students to that of both the on*campus and the 
HP students taking the same courses by live television. One result of this 
evaluation Is shown In Fig. 1, using data gathered during the autumn and 
winter quarters of the 1973-74 school year. This figure shows the grade 
point averages of the TVI students for all courses taken under the program 
plotted against an abscissa of admission qualifications, the latter parameter 
being derived from a composite of corrected undergraduate grade point average 
(UCGPA) and the quantitative component of the Graduate Record Examination 
score (GREQ). (The correction to the undergraduate GPA was based on prior 
history of performance of students from the same school in doing graduate 
work at Stanford.) Also shown 1i Fig. 1 are similar data for on-campus 
graduate students taking the same courses as the TVI students. The regression 
line of Fig. 1 shows the best linodr least-mean-square fit to the on-campus 
student data, from which the averago expected performance may be obtained 
for a student with given Input qualifications. It Is apparent that all of 
the TVI students performed significantly better than expected, based on the 
on-campus regression line. Wh1!e the admissions qualifications were on the 
average poorer than the typical Stanford Graduate Engineering student, the 
average performance In the courses examined was actually better than the 
On-campus average; In fact, the TVI students with the two lowest adnn^!;ion 
qualification scores would not have been admitted to the Stanford Graduate 
School based solely on these qualifications, but on the basis of their TVI 
performance have subsequently been admitted and have completed their MS degree 
programs with creditable graduate GPA performance. It. Is also useful to 
mention that for the first two quarters of the program the TVI students were 
required to come to the campus to take the same examinations as the on-campus 
students In the same physical setting. While this was occasionally a hardship 
for the TVI students. It did eliminate a number of factors that might other- 
wise have been used to minimize the performance of the TVI students (e.g.* 
failure to perform under competitive t%%t pressures). 

A second evaluation of TVt student performance during the first two 




quarters of the 1973-74 school year is shown in Fig. 2, in which the off- 
campus industrial videotape student grades are compared both with on-campus 
classroom students and industrial students watching the same courses by 
live television. To provide a basis for comparison, the 302 on-campus stu- 
dents taking the courses under evaluation achieved a grade point average 
(6PA) of 3.38 out of a possible 4.00. which is typical of graduate electri- 
cal engineering students at the Master's level. The industrial students 
participating in the same courses by live television with audio talkback 
capability to the classroom had a GPA of 3.19, still quite acceptable but 
significantly below the on-campus students. The limited sample of TVI 
students taking the !ianie courses by videotape with local tutors achieved 
an astonishing 3.67 GPA. The TVI students thus outperformed their Industrial 
counterparts, who had comparable job situations and academic backgrounds but 
were using live television, by almost half a grade point. They also outper- 
formed the on-campus students by alrrost a third of a grade point. This 
result is made even more remarkable v/hen we recall that several of the TVI 
students had marginal academic qualifications that would have made their 
admission to the Stanford graduate program very unlikely. It is also inter- 
esting to note that the performance of students at Santa Rosa in the videotape 
courses without local tutors was substantially below that of all the other 
groups. Although the data In this case was very limited. 

These results raise several questions that deserve some comment: 

(1) Clearly the total number of courses taken by students studying 
from videotape without a tutor is so small that the result has no statistical 
significance. However, a similar result has been obtained recently by 
Anderson [5] in a much larger experiment. In particular he finds that both 
student satisfaction and course performance tend to decrease as the delivery 
method is changed from on-campus lectures to live TV to non-tutored videotape. 

(2) It Is frequently argued that the Industrial experience of the TVI 
Sw/'ionts account-^ for their outstanding performance. However, the TVI stu- 
dents are drawn from the same population as the students studying from live 
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TV, whom they outperformed, so industrial experience and/or motivation 
cannot account for these results. Furthermore, experiments to be described 
shortly show that on-caiiipus TVI students also outperform on-campus students 
who attend the regular lectures. Hence the effectiveness of TVI is not limi- 
ted to Industrially-based off-campus students. 

(3) It is also natural to question the degree to which the results may 
be due to a Hawthorne effect. However, as will be Indicated in the next 
section, the Santa Rosa TVI students have continued their superior performance 
for three years. It seems somewhat unlikely that the Hawthorne effect can 
be made to persist for this period of time. And in any case, if the Hawthorne 
(or any other) effect can be used to improve performance over a full MS pro- 
grciii', then we have used it to our advantage. 

C. Continuation of the TVI Industrial Program . Because of the initially 
encouraging results with the Santa Rosa industrial TVI students. University 
permission was granted to contfnue the program and to incorporate TVI courses 
Into an accredited graduate degree program. New students at the Santa Rosa 
plant of Hewlett Packard entered the program In both 1974 and 1975. At tho 
end of the 1975-76 school year, after three years of operation* there have 
been 28 Industrial students at Santa Rosa who have taken a total of 1191 
quarter-hours of graded course work with an overall CPA of 3.34. Somv. of these 
students were interested only in occasional courses as a means of upgradlrijj 
their professional skills in particular areas and had either already completed 
or did not wish to pursue a graduate degree. Eighteen Santa Rosa TVI students 
have been admitted to degree candidacy, and of these ten have now completed 
their M3EE degrees with an overall CPA of 3.42. If we exclude from this 
group four students whose academic qualifications were below those normally 
required for Stanford admissions, the resulting CPA for the graduating Indus- 
trial TVI students at Santa Rosa Is an Impressive 3.59, considerably above 
the overall average for Stanford MSEE graduates with comparable backgrounds. 
The four formally "unqualified" students had a very respectable average GPA 
of 3.16. 

In the 1974-75 school year the TVI program was further expanded to 
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Include a Hewlett Packard plant in San Diego, and in the Spring quarter, 
the Hewlett Packard plant in boise, Idaho, adding some new dimensions to 
the program. In particular, the academic backgrounds and interests of the 
Industrial students at these locations were very different from the Santa 
Rosa students, who were all electrical engineering majors as undergraduates. 
By comparison, eight of the ten engineers taking courses at San Diego had 
their undergraduate degrees in such fields as Mechanical Engineering, 
Physics, Computer Science and Chemical Ln^^ineering. With such varied back- 
grounds and in some cases margi j1 undergraduate grades the role of the 
local tutor became especially important for locating and filling gaps In 
assumed prior knowledge. Perhaps For this reason the San Diego plant has 
made the most thorough use of the technique of stopping the videotape for 
active class discussion and participation, sometimes spending as much as 
two hours to complete the 50 minute lecture. Of the 10 participating stu- 
dents at San Diego, 7 have now been admitted to the Stanford MSEE program 
and to date have completed 87 quarter hours of course work with an overall 
CPA of 3.17. The Boise industrial students also have varied backgrounds and 
academic preparation; however the Boise plait is sufficiently new that the 
program has not yet reached the necessary level of stability to permit 
meaningful data to be gathered. 

The situation at the Sandia Corporation (Albuquerque) is also different 
from the three HP locations, and adds some new dimensions to the evaluation 
of TVI as ar n jctional methodology. Sandia has a highly developed in- 
house continu;.-^^ education program, and most of the professional engineers 
either have advanced engineering degrees or do not wish to pursue an MS.EE 
degree at Stanford. Instead, the program has been used as a continuing 
education opportunity in selected areas by mature professionals, usually 
only taking a few courses of particular interest, or in a few cases as a 
means of preparation for and entry into a Stanford Ph.D. program. Few of the 
Sandia students have requested formal admission to the Stanford graduate 
program; nevertheless, the results of the program seem to have been favorable. 
By the ned of the 1975-76 year a total of 423 graded quarter-units ha^e been 
completed by Sandia students with an overall 6PA of 3.58. 



17 



In sumnary, during the first three years of operation of the TVI 
program, a total of 1803 quarter-units of graded course work, plus a 
considerable number of ungraded seminars, have been completed by indus- 
trial students. Their overall CPA for this work is 3.37. Furthermore 
their response to the technique is very enthusiastic. On the basis of 
these results it seems reasonable to conclude that, for science and en- 
gineering courses, the TVI format is at least as good as the other methods 
of delivery with which it has been compared. It can be clearly superior to 
these other methods, despite inadequate background preparation on the part 
of the students, if the course tutor is carefully selected and effective 
group interaction is achieved. We believe the method can be used effective- 
ly for other types of courses though we have no data to support this hypo- 
thesis. We have, however, used the technique in two on-campus experiments 
to determine whether full time graduate students could also profit signifi- 
cantly from the TVI approach. These experiments will be described in the 
next section. 

D. On-Campus TVI Experiments . The first set of experiments was per- 
formed over two successive quarters in a graduate Electrical Engineering 
course that was large enough that several TVI groups (varying in size and 
with different tutors) could be formed for performance comparison with the 
large live lecture class. The course itself was taught by a faculty member 
who had already established a reputation as being a particularly effective 
instructor for both on-campus and industrial TVI students. Three TVI groups 
were formed, two of these being led by the same tutor. The tutors were 
chosen from a set of students who had performed equally well in the course 
during the previous year. Both tutors were very interested in the subject 
and were also interested in teaching as a career. However, their possible 
teaching styles were judged to be somewhat different. Where one tutor 
tended to answer questions directly when the tape was stopped, the other 
tended to encourag^e his group to find the answer themselves. The first 
tutor was given a group of six students (labeled Group III in the following 
Figures). The second tutor was given two groups of students, one with 
twelve members (identified as Group I) and one with six (identified as 
Group II). 
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At the outset it was thought that better controls in this case' would 
permit more sophisticated statistical analysis of the TVI comparative 
educational effectiveness. It was found, however, that for a variety of 
possible reasons (small sample size, variation in tutor effectiveness, use 
of volunteers in the TVI groups, etc.) significant overall improved educa- 
tional effectiveness could not be shown with high statistical reliability. 
However, as illustrated by Fig. 3, the data unequivocally support the hypo- 
thesis that the TVI method is at least as good as live instruction for all 
of the TVI students. Furthermore, for students with lower admissions quali- 
fications, the TVI method of teaching was found to be significantly more 
effective than in-class teaching. In fact the regression line for TVI 
students in Figure 3 suggests that the course performance is essentially 
independent of the standard measures of academic ability. This finding, 
which agrees with the TVI results obtained with the off-campus industrial 
students, becomes even more important in view of the fact that the "lower 
ability" group in this study had undergraduate GPA's in the range of 3.0 
to 3.3. It thus seems reasonable that students of "lower ability" may profit 
more from the TVI method of teaching, though this is to be expected to some 
degree because there is less room for improvement among the students of "high 
ability." 

Student opinions of the TVI format were also collected at the end of 
each quarter and showed a generally enthusiastic response to the method. 
Some characteristic features of their responses are shown in Figure 4, from 
which some general effects of tutoring technique and group size can be de- 
termined. With respect to the effect of group size, it is clear from a com- 
parison of Groups I and II (same tutor) that the smaller group was generally 
n.ore enthusiastic about the TVI experience than the larger group. This 
same general result has been repeated in our industrial TVI groups, and a 
consideration of all of our present data on group size leads us to the 
tentative conclusion that the method works best when the group size is be- 
tween 3 and 10. 

Figure 4 also contains some specific information about the influence 
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of tutoring stylos that Is consistent with our general observations. Spe- 
cifically the students studying with the tutor who tended to answer questions 
directly (Group III) were less well pleased than students In a group of equal 
size studying with a tutor who tended to draw them Into the discussion when 
the tape was stopped. Apparently, for on-campus TVI students, the opportunity 
to obtain direct answers from a tutor to more questions than could be asked 
of the In-class professor is not viewed as a major improvement, while the 
ability to discuss problems among themselves does represent a significant 
Improvement. 

The second set of experiments, performed during the Winter quarter of 
1976, was conducted with the second half of an introductory graduate engi- 
neering economics systems course (EES201B) in which the regular course 
professor was away from campus. In the absence of a suitable replacement, 
videotapes of his course from the previous year were shown to the regular 
class in the presence of another instructor. However, for technical and 
scheduling reasons, the tapes shown to the large class could not be stopped 
for discussion. 

Two small TVI groups were formed by random sampling from groups of 
students who had demonstrated high and low performance in the prior quarter, 
with each group having a separate tutor. A control group was also formejl, 
consisting of 17 students who watched the videotape in the presence of the 
professor, but had no opportunity to ask questions or make remarks during 
the showing of the tape. 

With respect to course grades, no statistically significant difference 
was found in the performance of the TVI and control groups, partly because 
of the small size of the sample groups, although again the data was consis- 
tent with the conclusion that the TVI students did at least as well as the 
regular class, and thdt the students of lower ability benefited more by the 
TVI educational method. In fact the comparison was frustrated by the fact 
that all low ability students in the control group dropped the course! 

However, a pronounced difference was observed in student attitudes 
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toward the use of a videotape with and without the combined tutor and tape 
►topping features. These results are shown in Figure 5, where the responses 
to several questions are recorded for the three groups of students ("low 
ibility," "high ability" and control). Each question was answered on a 
preference scale which ran from +2 (strongly agree) to -2 (strongly disagree), 
^ote that the "high ability" TVI group was highly enthusiastic on all counts, 
though their course performance was almost identical to what it had been 
during the previous quarter in the live lecture class . In fact the "high 
ibility" TVI group stopped the tape on an average of every five minutes. 

IV. OBSERVATIONS 

The tutored videotape instruction technique was invented in an effort 
to provide the benefits of both lectures and small group discussions to off- 
campus engineering students- Experience gained from three years of operation 
of the program suggests that the TVI technique is at least as effective as 
either classroom instruction or live TV with audio talkback capability, for 
both on-campus and off-campus students. However, our data is not yet suffi- 
ciently extensive to permit a rigorous statistical test of this conclusion 
to be made. We are also unable to generalize to subject areas other than 
engineering and science, though we believe the general principles of the 
TVI format will apply to a wide range of subjects and audiences. 

To assist the reader in considering how the method might be applied to 
situations other than those described above, it will be helpful to enumerate 
the factors which we believe to be critical to the effectiveness of the TVI 
format and to provide some estimate of program costs based on our operating 
experience. 

A. Factors that Contribute to the Effectiveness of TVI . Our experience 
suggests that the following factors arc important for the educational effec- 
tiveness of TVI: 

1. Tutor attitude, personality and Instructional style are very 
important. The tutor should be willing and Interested In helping the students 
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in his Group. He should attend all or nearly all of the TVI sessions. Tutor 
competence is Important, but it is better that he not be so over-qualified 
that he becomes bored or impatient with a lack of understanding in his stu- 
dents. Compensation for tutor effort is important for a continuing program. 

2. Group size is also very important. Fewer than tv/o or three stu- 
dents does not lead to good interaction and tends to make the method expensive, . 
while group size greater than 8-10 tends to inhibit discussion and reduce 

the frequency with which the tape is stopped. A group size of 3-8 seems 
optimum* although this can vary with student personalities and acquaintance 
with each other. 

3. Depending on the maturity of the student, committment to a degree 
program or similar educational objective may be important in sustaining interest 
and motivation. Certainly for most students completion of graded problems and 
examinations results in a more productive educational experience. 

4. Active classroom participation in the live class is desirable. 
For the subjects and audiences served to date, unrehearsed, unedited video- 
tapes of classroom lectures may be used and, in fact, may have more "prei,?nce" 
and be more interesting to watch than tightly-scripted, professionally-pro- 
duced lectures. 

5. It is important that the instructor be well-organized, knowledge- 
able In his subject, and free of annoyinq mannerisms. The charisma of a good 
Instructor is emphasized on the videotape. 

6. For Industrial students, management attitudes toward partlclpatloi 
play a very important role in the success of a continuing program. Job pres- 
sures which create long hours and interfere with family life markedly Increase 
the difficulty of pursuing an educational program. 

7. Continued management and evaluation of a TVI program needs to be 
the concern and principal responsibility of a person designated to coordinate 
and provide llason between the academic institution and the TVI students. Many 
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details require timely attention that would otherwise not be given by either 
the instructor or the company. 

None of these factors impose unmanageable requirements, but they are 
all important. With attention to group size, good tutoring, quality of 
recorded educational material, adequate handling of supplementary materials 
and grading of problems and examinations, the TVI methodology can provide an 
excellent educational experience and oppofrtunities for needed education in 
otherwise difficult or impossible situations. 

Use of the TVI format in a large scale application will, of course, 
bring special problems of its own, such as difficulties in recruiting and 
training tutors, organizing and supporting large numbers of local tutorial 
groups, reproducing and distributing the videotapes and homework, arranging 
for the grading of homework and tests, and so on. Clearly our experience 
cannot offer much insight into how these problems can beet be solved. However, 
the need for a separate administrative structure is apparent, since the grad- 
ing alone for a large program "lould not be readily integrated into the daily 
operation of an existing educational institution. 

VlUh regard to the large scale distribution problem it is, of course, 
natural to consider the use of a satellite communications system for the 
transmission of lecture material and homework, and for telephone-based confer- 
ences between tutors and faculty. For the delivery of lectures^ videorecorders' 
could be installed at the separate sites to tape the lectures as they are re- 
ceived. Use of non-prime time and slow scan techniques could reduce the cost 
of such transmission. It might also be advantageous for the on-site tutor 
to watch the lecture as it is being taped and to call the faculty member to 
discuss problems before he shows the tape to his TVI group. 

There is one cautionary note that should be added here, however. In- 
,tr1guing as the use of satellites might be. In the TVI program they are basi- 
cally replacing either the mall service or a closed circuit TV broadcast 
system that is operated In precisely ihe same way as the satellite system 
would be. Hence, there Is a question of what the most cost-effective method 
of delivering TVI might be. There Is also a question concerning the area 
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over which it is advisable to distribute TVI groups. In particular, if a 
course requires regular face-to-face contact between tutors, faculty mem- 
bers and coordinators, a maximum travel distance of perhaps 500 miles is 
possibly indicated. In these cases the use of a satellite, which offers 
coverage over much larger distances, may actually encourage the distribu- 
tion of TVI grou' s over a larger area than is desirable. 

B. Equipment jnd O perating Costs Obtained from the Experimental Pro- 
gram . We* conclude with a brief summary of the equipment and operating costs 
that have bee^^ obicii Kd from an analysis of the experimental program operated 
at Stanford. More exliMi'iive cost data on the Stanford ITFS System is avail- 
able in Miller and Baldwin [6]. 

1. Capital Equipment . Expenses for capitcil equipment can be 
divided into those associated with the classroom and those associated with 
the off-campus viewing facility. The classroom modifications that are neces- 
sary have been described in detail by Pettit and Grace [4] and include the 
provision of an overhead TV camera above the lecture platform, a camera 
mounted at the rear of the studio classroom, and appropriate lighting and 
air conditioning. These classroom modifications cost approximately $25*000. 
Control equipment and two black-and-white half-inch reel-to-reel videorecor- 
ders (one for back up) cost an additional $20,000, for a total of $45,000. 

At each remote installation, two videotape players {one for back up) 
and a 17" TV monitor are set up on a cart that can be rolled into any of 
a number of viewing rooms at the off-campus site. The complete cost of 
this installation is somewhat less than $2,000. 

2. Operating Costs . The operating costs for the experimental 
program will be stated on a per lecture basis; 
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ITEM 



AMORTIZED COST/LECTURE 



1. 1/2" tape 0 $16. played $ 1 00 
twice per lecture 

2. Videotape players and recorders 3.40 
(amortized over 10 years) 

3. Student operators 3. SO 

4. Tape mailing (preparation ami 5.60 
round trip postage) 

5. Tutor salaries 'for on-campus 3.50 
teaching assistant) 

6. Homework grading expenses 2.00 

7. Engineering and administration 1.50 

8. Space amortization ;_50 



$ 22.00/1 ecture 

The lectures can therefore be provided to a group of 10 TVI students 
at a cost of $2.20 per student-lecture. For rrvnparlson, the operating cost 
of "real time" television using the Stanford ITFS i^^stem is $2.43/student- 

It should be noted th;>t ..oth the live television and the TVI operating 
^ost? girg ni:^r j^nal costs; i.e., the cost of operating the on-campus class 
UiClf Is not included. 

V. CONCLUSIONS 

In conclusion. It seems worthwhile to summarize the principal, advantages ^ 
which we believe TVI has over both In-class instruction and live television. 
These are as follows: 

1. It Is (or can be) educationally more effective than either classroom 
or televised Instruction, especially for marginally qualified students. This 
Is primarily because It permits students to a^k more questions and, through 
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organized discussion, find more answers for themselves than they could in 
cither of the otiier formats. 

2. It is cheaper than either classroom or televised instruction from 
the point of view o\ marginal costs. This Is because it requires neither 
new educational plant nor expensive broadcast facilities. 

It makes good use of teaching resources by using faculty for course 
preparation and paraprofessional tutors for discussion of the lectures. For 
this latter function it draws on the substantial (if latent) interest in tu- 
toring that exists in a large segment of the population- 

4. It allows instruction to take place at the convenience of the students 
It is not bound by either the academic calendar or a broadcast schedule. 

These advantages are substantial, and while over-generalizing is dangerous 
we nonetheless feel confident that TVI can be successfully extended to larger- 
scale applications, at least for courses in science and engineering. We be- 
lieve the method can also be successfully extended to other subject areas. 
Such extensions will undoubtedly bring special problems of their own, but the 
probability of success seems high enough to warrant the effort of trying. 
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APPENDIX 



USE OF TKF GUlCEL li;:S F OR AN ANALYSIS OF SESA VI STREET 
_ AND THE CHICAGO TV COLLEGE 

The purpose of this appendix is to elaborate briefly on th? guidelines 
that were described in Section II. For this purpose we will use the guide- 
lines to analyze two very different ventures, Sesa.^e Street and the Chicago 
TV College. For Sesame Street [7,8,9]: 

(1) Target Audience . The program was designed primarily for inner-city 
pre-school children fron three to five years of age. It has of course found 
wide acceptance amongst a far more general audience; however, it seems likely 
that other specific audiences could be better served by a somewhat different 
program. 

(2) Educational Objectives . Instructional goals (viz., syribolic repre- 
sentation* problem-solving and reasoning* and familiarity with the physical 
and social environments) were carefully defined at the outset and were clari- 
fied for program writers through the Writers Workbook , in which each curri- 
culum objective is described and strategies for achieving it are offered. 

(3) Range of Media Employed . The major technology employed is broad- 
cast television, though non-broadcast materials (magazines, books, games) 
have been developed and the educational effectiveness of the program is 
increased markedly when they are used. 

(4) Kedia-sensitive Educational Staff . From the outset educational 
psychologists, specialists in children's education and television writers 
and producers worked together in the development and production of programs. 

(5) Personal Interaction . An attempt is made to structure personal 
interaction through the distribution of a Parent Teacher Guide to Sesame 
Street, to be used in homes, day care centers and kindergartens. However, 
this component of the overall strategy has been far less successful than 
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other aspects of Sesama Street. 



(6) Evaluation find Feedbac k. Individual programs viere produced and 
tested for educational effectiveness nearly one year before they were actual!: 
broadcast. When broadcasting began, children from the target audience were 
tested in major instructional areas at 3, 6, and 9 week intervals and the 
results were used to modify the remaining programs. 

For the Chicago TV College [8» 10]: 

(1) Target Audience . The program was designed primarily for a home- 
viewing, highly-motivated audience that could not attend courses offered on 
the college campus. 

(2) Educational Objectives . The audience was primarily interested in 
degree-oriented course work that was identical to course work already being 
presented on the college campus. 

(3) Range of Media Employed. Broadcast television is used for course 
delivery. Telephone conmunication with the instructor is available immediate- 
ly after the telecast. Course texts and other printed materials are also usee 

(4) Media-sensitive Educational Staff . Faculty who volunteer to teach 
on television are screened for academic excellence, classroom competence and 
television presence. Faculty who pass the preliminary screening are then 
trained by a TV production staff. 

(5) Personal Interaction . Personal interaction between the faculty and 
students has been an important factor in the success of the program. Regular 
telephone contact between faculty and students has increased student retentior 
In some courses from 2S% to SQ%. 

(6) Evaluation and Feedback . Evaluation is provided by (1) a continuing 
survey of audience needs and characteristics carried out by the college and 
(11) grading of homework and tests. One result of this continuing evaluation 
was that the telephone contact described above was Instituted. Intervals be- 
tween repetition of courses are also established In this way. 
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It will be clear from this comparison that the individual guidelines 
are not ecjually important in different situations nor do they fully describe 
the educational procjra:iis from v/hich thoy were derived. Nonetheless v/e 
believe they can help educational planners in a general way to improve the 
use of technology in education. In particular, they provide some means of 
assessing the time scale and cost required to niake tech ologicaTly-delivered 
education effective. For example, the Chicago TV Coll as organized to 
make efficient use of an established educational institution and established 
instructional techniques. It attracts an already-motivated audience v/ith 
known educational objectives- In such a situation the effective use of tech- 
nology can be developed quickly and at low cost; The College aired its first 
programs only 3 months after planning began. The total planning budget was 
approximately $300,000, and the average cost per course for enrolled students 
over the first 3 years of operation was only $75. 

For comparison, the production of Sesame Street involved the development 
of new educational techniques and a new form of collaboration between educa- 
tors and media experts. As a result, nearly 18 months were spent in the plan- 
ning, pre-production and testing of programs. The budget during these 18 
months alone v/as nearly $2 million. 

Before concluding this appendix, we also wish to elaborate further on 
the Importance of choosing the technology correctly. In this connection, it 
Is useful to mention the experience of WHA (state-owned and university-operated 
broadcast station in Wisconsin) with its program LET'S DRAW [11]. This program 
was designed to provide art education for children by radio broadcast. The 
program, v/hich had been highly successful on radio, was transferred to tele- 
vision when that medium became available. It was soon discovered that the 
program was much less successful on television than it had been on radio, 
largely because the students tended to copy the television artist. The program 
was then returned to radio and is still successful there. Here we have an 
excellent example of the importance of choosing the technology in terms of 
the material being presented. 
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flO. 2 r RESULTS OF TUTORED VIDEOTAPE INSTRUCTION EXPERIMlirr 
(October 1973 March 1974) / v 
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FIG. 3 - CLASSROOM AND TUTORED VIDEOTAPE INSTRUCTION STUDENT PERFORMANCE 
{EE 221A - Winter 1975) ^ 
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